Skip to main content

Logging Configuration Guide

This guide details the logging configuration options available in the Excel import system. Proper logging configuration is essential for debugging, monitoring, and maintaining the import process.

Overview​

The logging system provides granular control over which components generate logs. Each component's logging can be enabled or disabled independently through the LOGGING_CONFIGS section of the configuration.

Configuration Structure​

All logging configurations are boolean settings ("true" or "false") that control logging for specific components:

{
  "LOGGING_CONFIGS": {
    "APPEND_OUTPUT_CONTROLLER_LOGGING": "true",
    "PROCESSOR_RULE_LOGGING": "true",
    // ... other logging configs
  }
}

Available Logging Options​

Input Processing Logs​

  • EXCEL_READER_LOGGING: Logs raw Excel file reading operations
  • EXCEL_INPUT_CONTROLLER_LOGGING: Logs input validation and preprocessing
  • EXCEL_TABLE_LOAD_LOGGING: Logs table structure loading operations
  • EXCEL_MODULE_LOGGING: Logs general Excel module operations
  • EXCEL_TABLE_CONFIGURATOR_LOGGING: Logs table configuration processing

Layout Processing Logs​

  • LAYOUT_TABLE_PARSER_LOGGING: Logs table layout parsing operations
  • SHEET_DIMENSION_EXTRACTOR_LOGGING: Logs sheet dimension detection
  • TABLETYPE_PARSER_LOGGING: Logs table type determination

Feature Processing Logs​

  • FEATURE_SEARCH_LOGGING: Logs feature search operations
  • FEATURE_CONTROLLER_LOGGING: Logs feature control operations
  • FEATURE_LABEL_CONTENT_LOGGING: Logs feature label content processing
  • FEATURE_SEARCH_CRITERIA_LOGGING: Logs search criteria evaluation

Cluster Processing Logs​

  • CLUSTER_CONTAINER_LOGGING: Logs main cluster container operations
  • CLUSTER_CONTAINER_LOGGING_SEARCH_ORDER: Logs cluster search order processing
  • SUB_CLUSTER_CONTAINER_LOGGING: Logs sub-cluster processing

Rule Processing Logs​

  • PROCESSOR_RULE_LOGGING: Logs rule processing operations
  • RULE_LOAD_CONTROLLER_LOGGING: Logs rule loading operations
  • RULES_CONDITIONS_PROCESSOR_LOGGING: Logs rule condition evaluation
  • FINAL_ROW_FILTER_PROCESSOR_LOGGER: Logs final row filtering operations

Log Output Configuration​

The logging output is controlled by several general configuration settings:

{
  "GENERAL_CONFIGS": {
    "USE_LOGGER": "true",
    "MINIMIZE_LOGGING": "true",
    "VERBOSE_LOGS_SUFFIX": "verboseLogs.csv",
    "USER_LOGS_SUFFIX": "userLogs.csv"
  }
}

Output Files​

  • verboseLogs.csv: Contains detailed technical logs
  • userLogs.csv: Contains high-level user-friendly logs

Best Practices​

  1. Development Environment

    • Enable comprehensive logging during development
    • Use FEATURE_SEARCH_LOGGING and FEATURE_CONTROLLER_LOGGING for debugging feature issues
    • Enable CLUSTER_CONTAINER_LOGGING for understanding data grouping

  2. Production Environment

    • Minimize logging to essential components
    • Enable FINAL_ROW_FILTER_PROCESSOR_LOGGER for tracking data filtering
    • Keep EXCEL_READER_LOGGING enabled for input validation

  3. Debugging Specific Issues

    • Enable relevant component logging based on the issue area
    • Use VERBOSE_LOGS_SUFFIX for detailed investigation
    • Keep MINIMIZE_LOGGING set to "false" during debugging

Troubleshooting Common Issues​

Missing Data Issues​

Enable these logs:

{
  "EXCEL_TABLE_LOAD_LOGGING": "true",
  "FEATURE_SEARCH_LOGGING": "true",
  "FEATURE_CONTROLLER_LOGGING": "true"
}

Layout Detection Problems​

Enable these logs:

{
  "LAYOUT_TABLE_PARSER_LOGGING": "true",
  "SHEET_DIMENSION_EXTRACTOR_LOGGING": "true",
  "TABLETYPE_PARSER_LOGGING": "true"
}

Rule Processing Issues​

Enable these logs:

{
  "PROCESSOR_RULE_LOGGING": "true",
  "RULE_LOAD_CONTROLLER_LOGGING": "true",
  "RULES_CONDITIONS_PROCESSOR_LOGGING": "true"
}

Performance Considerations​

  • Enabling all logs can impact performance
  • Use selective logging in production
  • Consider log rotation for long-running processes
  • Monitor log file sizes when extensive logging is enabled

Log Analysis​

Understanding Log Patterns​

Each log entry typically contains:

  • Timestamp
  • Component identifier
  • Operation type
  • Relevant data values

Example log pattern:

[Timestamp] [Component] [Operation] : [Details]

Common Log Messages​

Understanding common log messages helps in quick problem identification:

  1. Feature Search Messages:

    [Feature Search] Starting search for pattern: [pattern]
    [Feature Search] Found match at position: [position]

  2. Cluster Processing Messages:

    [Cluster] Processing container: [container_id]
    [Cluster] Found items: [count]

  3. Rule Processing Messages:

    [Rule Processor] Applying rule: [rule_id]
    [Rule Processor] Rule result: [result]